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Abstract 

We consider Parikh images of languages accepted by 
non-deterministic finite automata and context-free gram- 
mars; in other words, we treat the languages in a com- 
mutative way — we do not care about the order of letters 
in the accepted word, but rather how many times each one 
of them appears. In most cases we assume that the alpha- 
bet is of fixed size. We show tight complexity bounds for 
problems like membership, equivalence, and disjointness. 
In particular, we show polynomial algorithms for member- 
ship and disjointness for Parikh images of non-deterministic 
finite automata over fixed alphabet, and we show that equiv- 
alence is Uncomplete for context-free grammars over fixed 
terminal alphabet. 



1 Introduction 

We consider languages accepted by regular and context- 
free grammars, except that we treat the language in a com- 
mutative way — we do not care about the order of letters 
in the accepted word, but rather how many times each one 
of them appears. In this setting, usual problems, like mem- 
bership and equivalence, have different complexities than in 
the non-commutative case. 

A well known classic result in this area is the result of 
Parikh |Par66J that, for a context-free grammar G over al- 
phabet E, the Parikh image of G, i.e., the set out(G') C 
of such multisets M that, in some word w e L{G), each let- 
ter X appears M{x) times, is a semilinear set. Some com- 
plexity results regarding semilinear sets and commutative 
grammars have been obtained by D. Huynh IHu80llHu85l . 
who has shown that equivalence is Ilf -hard both for semi- 
linear sets and commutative grammars (where IIj is the 
dual of the second level of the polynomial-time hierarchy, 
UStoTTl -). 

There are many practical uses of regular and context-free 
languages which do not care about the order of the letters in 
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the word. For example, when considering regular languages 
of trees, we might be not interested in the ordering of chil- 
dren of a given node. |BM99l and |p4S99| consider XML 
schemas allowing marking some nodes as unordered. 

Some research has also been done in the field of 
communication-free Petri nets, or Basic Parallel Processes 
(BPP). A Petri net ( IPetSll . IRei85l ) is communication-free 
if each transition has only one input. This restriction means 
that such a Petri net is essentially equivalent to a commuta- 
tive context-free grammar. IYen96l shows that the reach- 
ability equivalence problem for BPP-nets can be solved 

in DTI ME (^2'^''° ^ . For general Petri nets, reachability 
(membership in terms of grammars) is decidable IKo82L 
although the known algorithms require non-primitive re- 
cursive space; and reachability equivalence is undecidable 
0Ha76l . Also, some harder types of equivalence problems 
are undecidable for BPP nets IHu94ll . See IEN93II for a sur- 
vey of decidability results regarding Petri nets. 

It turns out that, contrary to the non-commutative case, 
the size of alphabet is very important. In the non- 
commutative case, we can use strings a, ab, and abb to en- 
code a three letter alphabet {a, b, c} using two letters. Try- 
ing to do this in the commutative case fails, since two dif- 
ferent words ac and 66 are mapped to aa66 and a6a6, which 
are commutatively the same word. There is no way to map 
a three letter alphabet to a two letter one which does not 
collapse anything. Each new letter adds a new dimension to 
the problem in the commutative case — literally: commu- 
tative words (multisets) over an alphabet of size d are bet- 
ter viewed as points in a d-dimensional space, rather than 
strings. 

Contrary to most previous papers on commutative gram- 
mars, in most cases we assume that our (terminal) alphabet 
is of fixed size. As far as we know, there have been no suc- 
cessful previous attempts in this direction (except for the 
much simpler case d — 1 IHu84l ). Our methods enable 
us to obtain tight complexity bounds for most of the basic 
problems (like membership, inclusion, equivalence, univer- 
sality, disjointness) for both regular and context-free com- 
mutative grammars, over an alphabet of fixed size. In some 
cases, we provide algorithms for the special case d = 2, as 
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they are much simpler than the general ones. 



In Theorem 5.3 we show a polynomial algorithm decid- 
ing membership for regular languages, i.e., whether a multi- 
set (given as a binary representation) is in the Parikh image 
of a regular language (represented by the non-deterministic 
finite automaton accepting it). 



In Theorem 6.2 we improve upon Parikh 's result quoted 
above in two ways, assuming that the alphabet is of size 2. 
First, out(G) is produced as a union of linear sets with only 
two periods (whose magnitude is single exponential in size 
of G); second, these linear sets can be grouped in a poly- 
nomial number of bundles such that each bundle shares the 
pairs of periods used (we call such a bundle an A, B-frame). 
This leads to a IIj algorithm for solving inclusion (equiva- 
lence) for context-free languages. Unfortunately, such sim- 
ple presentation is impossible for alphabets of size greater 
than 2; we provide a counterexample where d = 3, and a 
much more complicated reasoning which solves the general 
case (still resulting in a 112 algorithm). 

The following table summarizes our results. Alphabet 
size F means that alphabet is of fixed size, and U means un- 
fixed size. We consider the basic problems: membership, 
universality, inclusion, and disjointness; note that solving 
inclusion is equivalent to solving equivalence — simple 
reductions exist in both ways. We use c as an abbrevia- 
tion for complete. Our main results — our most impor- 
tant algorithms — are marked with bold (polynomial algo- 
rithms for checking membership and disjointness for regu- 
lar grammars over alphabets of fixed size, -completeness 
of the inclusion (equivalence) problems for context-free 
grammars over alphabets of fixed size). Problems which 
have been shown to be hard are marked with stars (NP- 
completeness of membership checking for regular gram- 
mars over alphabets of unfixed size and context-free gram- 
mars over alphabets of size 1, coNP-completeness of uni- 
versality checking for regular grammars over alphabets of 
size 1, IIj -completeness of inclusion (equivalence) check- 
ing for context-free grammars); the proofs are simple and 
have been included for sake of completeness. 
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2 Overview 

In this section, we present our techniques and results in 
an informal way. The formal version can be found in the 
following sections. 

Our main observation is that we can treat our runs (or 
derivations in CF grammars) completely commutatively: 
we just count how many times each transition (rule) has 
been used. In both cases, the validity of such a „commuta- 
tive run" can be checked by checking two very simple con- 
ditions: Euler condition (each state is entered as many times 
as it is used) and connectedness (there are no unconnected 
loops) — Theorem 4. 1 From this, we immediately get that 
checking membership of a given multiset in a Parikh image 
of a context-free language is in NP. 

The second observation is that we can decompose a 
run into smaller parts, ultimately obtaining its skeleton, to 
which we add some (simple) cycles. Since the skeleton uses 
all states that the original run used (which we call its sup- 
port), we can add these cycles in arbitrary numbers to our 
skeleton, always getting valid runs. Moreover, in case of fi- 
nite automata (regular grammars), both the skeleton and the 
cycles are bounded polynomially (in the size of the automa- 
\G\ — Lemma 5.1 



ton. 



Now, linear algebraic considerations come into play. 
Whenever we have d linearly independent vectors 
vi, . . . ,Vd with integer coordinates in a d-dimensional 
space, for each other vector v, there is a constant C such 
that Cv can be written as a linear combination of vi, . . . ,Vd 
with integer coefficients (Lemma [3. This C is bounded 
polynomially by coordinates of vi, . . . ,Vd {d appears in the 
exponent). In our case, our vectors will be the Parikh im- 
ages of our cycles, with d letters in our alphabet. 

Thus, whenever we have a non-negative integer combi- 
nation of more than d cycles, where the multiplicities of 
cycles are big enough, we can reconstruct our Parikh im- 
age using different multiplicities of these cycles, and do 
such ,, shifting" until the multiplicities of some cycles drop 
(Lemma |3.2| and Theorem |5.2| i. Thus, there are at most d 
cycles which we are using in large quantities. From this, we 
get an algorithm for membership (Theorem 5.3 1: we can 
guess the small run and the d cycles (making sure that the 
run crosses these cycles), and then just check whether we 
obtain our result by adding the cycles in non-negative inte- 
ger amounts to our run — which boils down to solving a 
system of equations. This algorithm is polynomial, since 
everything it uses is bounded polynomially. 

The situation is the simplest in the case of d = 2, where 
instead of guessing the two cycles, we can always use the 
two extreme ones Wa, ^6 — i.e., the ones which proportion- 
ally contain the greatest quantities of the two letters a and 
b in our alphabet. Each other cycle can be written as a 
non-negative combination of these two. Still, we have to 
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take the extreme cycles which cross our small run — which 
means that we have to guess the two states where they cross, 
and then take extreme cycles crossing these states. For un- 
fixed d, or for context-free grammars, the problem is NP- 
complete. 

Now, what about context-free grammars? Generally, we 
can use the same techniques, but now, skeletons and cy- 
cles are bounded exponentially. In case of d = 2, we get 



Theorem 6.2 a Parikh image of G is a union of a polyno- 
mial number of A, B-frames; where an A, i?-frame (Defini- 
tion|3.3 1 is a set of vectors defined by some W (a subset of 



N'* bounded by A) and two vectors Va, Vb (bounded by B), 
consisting of all vectors of form w + UaVa + ribVb (where 
w € W). The number of A, _B-frames is polynomial, be- 
cause our two vectors will always correspond to extreme 
cycles from some two states. A and B are exponential. 

The following picture shows geometrically what an 
A, i?-frame is: a set W sitting inside of a box of size A 
(drawn as the letter W) is copied by shifting it in two direc- 
tions. The vectors by which we are shifting are bounded by 
B. 





It turns out that two such unions of A, _B-frames are 
equal iff they are equal in the exponentially bounded re- 
gion close to (Lemma |3.4| and Lemma |3.5| l. Together 
with the fact that membership checking is in NP, we get a 
IIj algorithm for checking inclusion (equivalence) of Parikh 
images of context-free grammars (for d = 2). 

For d > 2, it may be impossible to get a polynomial 
number of A, B-frames (a nice counterexample in Section 
|7]l, which means that Lemma [33] fails (the region would be 
bounded double exponentially). However, we can circum- 
vent this by splitting N'' into regions — when restricted to 
a single region, the number of A, B-frames will be poly- 



nomial, thus allowing us to use Lemma 3.5 successfully in 
each region separately, again getting a 112 algorithm for de- 
ciding equivalence. 

3 Geometry of Multisets 

For a set X, the elements of are interpreted as mul- 
tisets of elements of X. A set is interpreted as a multiset. 



thus for Y C X, Y{x) = 1 for a; G F and Y{x) = oth- 
erwise. For X E X and v E , x E v denotes v{x) > 0. 
We will sometimes write x instead of {x} (a multiset con- 
taining only a single occurence of a; e X). is also 
treated as a subset of , , and M^. For v E , 

= J2xex 1^(^)1' ll^ll = maxj^gx \vix)\. We say u > u 
iff Ux > Vx for each x. 

By we denote the set of matrices with coefficients in 
F and dimensions indexed with elements of X: for a matrix 
M E F^ , AP, the i-th column of the matrix, is a vector in 
F^ . For M E and v E Q^, Mv is a vector given by 
{Mv), = and ||M|| = max.gx ||M-||. 

We use the notation [0..iir] for the set of integers from 
to K, P for the set of non-negative rationals (we don't 
use the more standard notation of Q+ to avoid double upper 
indexing, as in (Q^)'^), and [0; K] for the set of rationals 
from to K. Thus, for example, [O-.i^T]^ denotes matrices 
with coefficients in N bounded by K. 

We can add or multiply sets of scalars, vectors, or matri- 
ces, in the usual way. For example, U + V = {u-\-v:uE 
U,v E V}, and MN^ for M E is the set of vectors 
which can be obtained as a linear combination of columns 
of M with coefficients from N. 

Lemma 3.1 Let M be a non-degenerate matrix in Z|<, and 
let V E Z^. Then (detM)v E MZ^. 

Proof. For vi,V2 E Z^, we say that vi = V2 iff 
vi — V2 E MZ^. The quotient group ZP 1= has detM 
elements (intuitively, for = 2, the number of elements 
is equal to the area of the parallelogram given by columns 
of Af; this intuition also works in other dimensions). Thus, 
(detAf)i; = 0. □ 

Lemma 3.2 Let V C [O-.i^T]^ be a linearly dependent set 
of vectors. Then for some a E ZX we have '^v'v — 0, 

where \a\ = 0(Xl^l|E|!), and > Ofor some V. 

Proof. Without loss of generality we can assume that 
V is a minimal linearly dependent set. Thus, we get 
J2vev /^^"^ ~ ^ ^'^^ some rational coefficients (3 E . 
Let u be such that > for each v. Let M E Z^ be a 
non-degenerate matrix whose |F| — 1 columns are V — {u} 
(we obtain a non-degenerate matrix since V was a mini- 
mal linearly dependent set; if |V^| < |S| + 1, we fill up the 
remaini ng c olumns with independent unit vectors). From 
Lemma 
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we get that (det M)u — Mw for some w E ZP . 
Let a„ = — dct M, ay = Wi where v — AP, and a„ = 
for remaining vectors. We have that ^ ayV — 0. Moreover, 
we have that for some q E Qwe have /3y = qa^ for each v 
(for a minimal linearly dependent set, (/?„) is unique up to 
a constant); thus, |a„| < = detAf = ©(i^l^l for 
each i. □ 
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Definition 3.3 An A, i3-frame is a set of form W + MN^, 
where C [Q..A]^, and M, G [O..S]|. 

Lemma 3.4 Let S = {a, 6}. Let A, B and C be positive 
integers. 

For w e and M in [O..B]|, let 

angle(w, M) = w + MV^ . 

For V G P^, the region ofv is defined as 

Vw e [0;AfyM e [O..B]| 
reg(u) — eV^ : w G angle(w, M) 
u G angle(ti;, M) 

For each v G N^. there exists a vector v' G such that 
\\v'\\^ 0{AB^ + BC), v' G reg(w) and v - v' G CZ^. 

Proof. The following picture shows this lemma graph- 
ically for B — 3. A is the size of the black square in the 
bottom left corner. Lines shown on the picture are bound- 
aries between angles; in each bundle, 6 lines are shown, 
but it should be understood that there is actually a semi-line 
starting from each rational point in the black square. 




Each angle is the set of points between two semi-lines 
on the picture which cross somewhere in the black square. 
There are three type of regions: ones containing only one 
vector (each bounded region is actually a singleton), 8 un- 
bounded regions in angular shapes (between two consective 
bundles of lines — there are 9 bundles of lines because vec- 
tors in [0..3]^ go in 9 directions), and regions in shape of 
semi-lines. 

It can be easily calculated that each point where bundles 
of lines going in different directions cross has its coordi- 
nates bounded polynomially (by AB^). 

Let w be a point. If reg(u) is a singleton, then we are 
done (because v is already bounded polynomially). Oth- 
erwise, reg(w) is the inside of angle(ti;, M), where w is 



bounded polynomially and M is given by vectors in two 
directions and AP (consecutive or equal). If v is in- 
side the paralellogram whose vertices are w, w + CM^, 
w + CJVP, w + CAP + CAP, then we are done (all those 
vertices are bounded polynomially). Otherwise we subtract 
multiplicities of CAI^ and CAP until we get a point w' in 
this paralellogram. □ 

Lemma 3.5 Let E = {a, b}. For i £ I, let Zi — Wi + 

AliN^ be an A, B-frame. Let v G N^. Then there exists a 
v' G such that \\v'\ \ ^ 0{{A + B)O(I^I)), and, for each 
i, w G Zi iffv' G Zi. 

Proof. Assume that the matrices Ali are non-degenerate 
(the case of degenerate matrices can be solved easily by 
changing the matrices). 

Let C be the least common multiple of determinants of 
matrices Mi, C = 0(B0(I^I)). 

Let V G N^. Let v' be the vector v' from Lemma [s^l for 
our V and C; we get that ||i;'|| = 0{{A + B)'='^W)), We 
will show that it satisfies our conditions. 

It is enough to check whether v G Z' iff v' G Z' for each 
Z' of form wq + M^N^, where wq G W.^. 

Since Ali is non-degenerate, for some a, a' G we 
have V = Wo + Alia and u' = + Alia' . Since v' is in 
the same region as w, a > iff a' > 0. On the other hand 
w - -y' G CiP C {deiAIi)lP C M^Z^ from Lemma 
Thus, V G iff w' G N^. 



3.1 



□ 



4 Commutative Grammars 

Since in this paper we don't care about the order of sym- 
bols in strings generated by our grammars, we define our 
grammars commutatively: a state (nonterminal) produces a 
multiset of letters and states, not a string. 

Derivation trees are defined for commutative grammars 
similarly as for the usual ones; we omit this definition. 
However, we usually also abstract from derivation trees, 
by considering our runs as multisets rather than trees: we 
don't care where in the tree each transition (production) has 
been used, we just count the total number of occurences. 
We show that there is a simple condition which checks 
whether our multiset corresponds to some full derivation, 
or a „cyclic" derivation. (A similar algebraic definition of 
cycles is used by the algebraic topologists.) 

A commutative grammar is a tuple G ~ (S, S*, sq, 5), 
where E is a finite alphabet, is a finite set of states, so G 

5 is an initial state, and (5 C S* x x is a set of 
transitions. We will write transitions {s,a,t) as s A t; in 
terms of derivations, each transition consumes the state s 
and produces each letter from a and each state from t. For 
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a transition t = s t, source(T) — s, target(T) = t, and 
out(r) — a. 

We will assume that each state is a source of some tran- 
sition. We will also assume that for each s A t G (5, 
\a\ < 1 and \t\ < 2. (We do this because we want to limit 
things produced by the grammar in terms of jS*!. Grammars 
not satisfying these conditions can be easily transformed 
by adding additional states.) A commutative grammar sat- 
isfying \t\ < 1 is called a regular commutative gram- 
mar (regular grammars are equivalent to non-deterministic 
finite automata, with initial state sq and transitions with 
target(r) = as transitions to the final state; we prefer 
to speak about regular grammars rather than NFAs for the 
sake of uniformness). 

For a D E N^, sourcc(_D) G counts how many each 
state appears as source of a transition: source(D)(s) = 
'Er:sourcc(r)=sD{T), and out(i:>) and target{D) counts 
how many each letter and each state, respectively, is 
produced: out(£') — X^r -^(''')*^^*(''')' target(D) = 
J2t ^(T)target(r). The support of D, supp(£)) = {s G 
5 : s G source(il')}. We say that D is connected from 
s G S" if for each t G supp(£') there is a path from s to 
t in D, i.e., a sequence ri, . . . , r,„ such that Ti E D, s = 
source(ri), source(Ti+i) G target(Tj), t E target(r„i). 
We say that D is a cycle from s E 5 iff it is connected 
from s and it satisfies the Euler condition: sourcc(Z?) — 
target (£>) (in terms of derivations, each state is consumed 
as many times as it is produced). We say that D is run iff it 
is connected from sq and source(£') = target(Z?) + {sq} 
(each state is consumed as many times as it is produced, 
except So which is consumed one time more). 

For a commutative grammar G, out(G') = {out(Z?) : 
D is a run in G}. 

The relation between algebraic runs and cycles and 
derivation trees is as follows: 

Proposition 4.1 Let G be a commutative grammar. Then: 

• D is a run iff there is a derivation tree fr'om Sq where 
each transition t appears D{t) times, and all the 
branches are closed, 

• D is a cycle from s iff there is a derivation tree from 
s where each transition r appears D{t) times, and all 
the branches are closed except one with state s at its 
end (we call such derivation tree cyclicj. 

Proof of Proposition |4.1[ We show the proof for runs 
(for cycles the proof is similar). 

Start with sq and try applying transitions from D (ob- 
viously, using each transition as many times as it appears 
in D) as long as we have some open branches. If we have 
used all the elements of D in the process, we are done. Oth- 
erwise, since the run D is connected, there must be some 



state s such that D contains some transition from s which 
is still not used, and s already appears in our derivation 
constructed so far. Since each derivation uses each state 
as many times as it was produced, and so does D, there also 
must be a yet unused transition ri in D which produces s, 
from, say, si. If s ^ si, for the same reason there must 
be a yet unused transition T2 in D which produces si from 
some S2- Finally, we produce some Sk from s. We create a 
cyclic derivation tree with transitions r^, . . . , ti on its main 
branch, closing all the side branches with remaining unused 
transitions from D. We insert this cycle into our tree (we 
have produced s in some place; we cut off the part of tree 
from this s, insert our cycle here, and we attach the part of 
tree we cut off to the open branch of our cycle). Repeat until 
all elements of D have been used. □ 

Thus, if G is a commutative version of some context-free 
grammar H, then out(G) equals the Parikh image of L{H), 
i.e., V G out(G) iff there exists aw E L{H) such that each 
letter a appears in w v{a) times. 

One inclusion is obvious. In the case of regular gram- 
mars, the cycle is just what is expected (a cycle in the tran- 
sition graph), and the other inclusion is equivalent to the 
classic theorem of Euler (characterization of graphs with 
Eulerian paths and cycles); in general, it is a simple gener- 
alization. 

A cycle is called a simple cycle iff it cannot be decom- 
posed as a sum of smaller non-zero cycles, and a run D is 
called a skeleton run if it cannot be decomposed as a sum 
of a run Di and a non-zero cycle G, where supp(£'i) = 
supp(-D). For each state s E S, let Cs be the set of simple 
cycles from s, and Ct = Usgt^s, for T C S. Also, let 
ys = out(Cs), and 3^t = Usgt (cycle outputs). 

5 Membership checking 

Lemma 5.1 Let G be a regular commutative grammar, and 
D be a run such that \D\ > 1 + n\S\. Then D — Di + nC, 
where Di is a run, C is a simple cycle, and supp(£'i) — 
supp(D). 

In case of n — 1, we get a limit on the size of a skeleton 
run. 

Proof. Let r be a transition such that D{t) > 1 + n\5\. 
We have |target(r)| = 1 (it cannot be greater because G 
is regular, and cannot be becuase each run in a regular 
grammar has exactly one transition with | target (t)| = 0). 
Let t E target(r). Let T be the set of states which can 
be reached from t via a path using only transitions such 
that D{Ti) > n. If source(T) G T, this finishes the 
proof (we have found a cycle in the graph of transitions, 
which can be easily translated to an algebraic cycle). Oth- 
erwise, let U = EreD:sourco(r)^T,targct(r)eT D{t), and 
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« = EreZ):sourcc(r)^T,targct(r)eT^(^)- From the Euler 
condition, we get that u ~ v. Since r is counted in u l+n\S\ 
times, and there are |(5| transitions, there must exist a transi- 
tion t' which is counted n + 1 times in v. This is a contra- 
diction (we have found a path from t to targct(T') ^ T). 

□ 

Theorem 5.2 Let G be a regular commutative grammar, 
and K G N^. Then K G out(G') iff there exists a run D in 

G, \\D\\ = 0(|5|^'^'|S|!), and simple cycles Ci,..., Cm, 
Gi G Csupp(£)), such that Ci are linearly independent and 



K = out(_D) + J2i a^out{Ci) for some a^, 



,a" 



Proof. 

Let Dq be a run in G such that K = out(Z?o)- We 
decompose the run Dq into a sum of simpler runs (on 
the same support) and simple cycles, until we get Dq = 



D2 + yZncr IcC, where D2 is a skeleton. From 

Lemma 5.1 we get that II-D2II < |<^|- By taking out's, we 



^yY, where 7^ G N 



get K = out (Da) 
for each Y . 

Let P C Xupp(Do) be the set of such cycle outputs Y 
that7y > LforL = Od^pllS]!). We can decompose 
K so that P is linearly independent. Otherwise, by Lemma 
3.2 X^rep otyY = for some ay, jay | < L, and ay > 
for some y . This allows us to transfer multiplicites between 
different cycles: if we take 7y = 7y — ay for Y £ P, and 
7^ = 7y for Y ^ P,v/e have = J2c 7(7^- We 

transfer multiplicites (i.e., replace 7 with 7') until one of 
our cycles is no longer in P. 
Let i?i = Da + E 



cec, 



supp(r>i ) 



_p^cC. Since IIZ52II < 

I S I 

|^|; 7c < and there are at most OdS*] ) distinct simple 
cycles in 3^supp(z?o) ^'^ equivalence of out's, we get that 



HAH < |<5|+LO(|5|l^l). Now,i^ = A+Ey6P7F>" 



□ 



Theorem 5.3 For an alphabet S of fixed size, and a com- 
mutative regular grammar G over E, and K G N^, the 
problem of deciding whether K G out(G) is in P. 



Proof. The theorem [572l remains true if we define Cs and 
3^s using short cycles instead of simple cycles — a cycle C 
is short iff |C| < IS"]. This allows us to calculate sets for 
each state S using simple dynamic programming. 

For each T C 5 of size at most |S]|, we calculate the set 
of possible out(Z?) with D satisfying the limit from Theo- 
rem 5.2 and T C supp(£'). For each element of out{D) 



and each sequence of linearly independent elements of 
Yi . . . Yjn, we check whether K ~ out{D) + Q^*^i for 
some a'\ which can be done by solving a system of equa- 
tions. □ 



Theorem 5.4 For an alphabet S affixed size, and two com- 
mutative regular grammars G and H over S, the problem 
of deciding whether out(G') n out{H) = is in P. 



Proof. Note that in the proof of Theorem 5.3 we have 



actually never used our assumption that outputs of our tran- 
sitions are non-negative, e.g.. Lemma [J!2| works as well for 
V C [—K..K]^. Thus, we can check whether out(G) and 
out{H) are disjoint by checking whether ^ out{GH^^), 
where H^^ is obtained from H by negating outputs of all 
transitions, and GH~^ is a regular grammar obtained via 
the usual method of concatenating languages given by reg- 
ular grammars G and H^^ . □ 



6 Inclusion checking 

Lemma 6.1 Let G be a commutative grammar over S. If 
D is a simple cycle or a skeleton run, then out (I?) — 

Proof. We start with the cycle case. We consider its 



cyclic derivation tree from Proposition 4. 1 



If somewhere on the branch leading to s (the main 
branch) we had another s, we can easily split our cycle into 
two cycles (by splitting the derivation tree). A similar thing 
can be done if we had some state t in two places on the main 
branch. 

A similar operation can be done when we find the same 
state twice on the side part of a branch (i.e. the part disjoint 
with the main branch). 

Since we can use each state at most twice on each branch 
(once on the main part and once on the side part), this limits 
the size of a simple cycle to exponential in size of G. 

The construction for skeletons is similar Indeed, con- 
sider a skeleton run. If a state s appears 15*1 + 1 times 
on a branch of a production tree, it means that there exist 
two consecutive appearances of s such that the part of tree 
between them can be cut off without removing any state 
from the support of this skeleton (otherwise each such state 
would have to be different and we would have l^l + 1 states 
in total). □ 

Theorem 6.2 ("normal form") Let S = {a, b}, and G be 

a commutative grammar over S. Then out(G') = Eie/ 
where \I\ — \S\^'^^\ and Zi are A, B-frames, where 



Proof. 

Let Z? be a run of G, and T = supp(i:>). For Y G 3^t, 
let b{Y) ^ Yib)/\Y\; let y° and Y'> be the elements of J^t 
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with the smallest and largest b{Y), resp ectiv ely. We have 
\Y''\ O (^21^1''*") from Lemma [elj There are at 

most |S'|2 possible pairs {¥",¥''). Let R{Y'',Y'') be the 
set of runs having particular y° and Y''. We will show that 
out(i?(y, Y'')) is of form W + MN^, where the columns 
ofMarey^ and y^ 

We decompose D as D2 + J^ceCr '^C'C, where D2 is 
a skeleton. Thus, out(I?) is decomposed as out(Z32) + 
Ereyr'^yY + P-^" + PbY\ where Pa.Pb^ay e N. 
We can assume that each ay < det M, because other- 
wise we can replace (detM)y by QaY'^ + qtY^, where 
QajQb G N (the coefficients are integers from Lemma 3.1 
and non-negative since and Y^ are extreme cyc les). 



from Lemma 



6.1 



and 



Each Y and out(i:'2) is O (^2l^l°" 
there are O (2^'^^'^^ 

By taking for W the sets of possible D3 for all runs from 
Y^), we get the required conclusion. □ 



possible Y's, thus ~ out(Z?2) 



Theorem 6.3 Let Gi and G2 be two commutative gram- 
mars over S = {a, 6}. Then the problem of deciding 
out(Gi) C out(G2) is -complete. 



Proof. The problem is IIj -hard because we can reduce 
the problem of semilinear set inclusion I HuSOII to it. 



Using Theorem 6.2 we can write each out(G/c) as 
Uie/fe where is a polynomial set of indices and 

is a A, i?-frame, where A and B are O ^2l'^l'^*^' ^ . 

From Lemma |3.5| we get that it is enough to check in- 
clusion on vectors v e of size \v\ < P = 0{{A -\- 
B'jO{\ii\+\i2\}y \Ye call such vectors small vectors. 

A witness for membership of w in a grammar G is a run 
D such that out(I?) = v, and < P. If u is small, and 
D does not contain non-productive cycles (i.e., G such that 
out(G) = 0; such cycles can be eliminated), then it can be 
described as a string of length polynomial in size of G. We 
call such witness a small witness. 

For each v, and each small witness of membership of v 
in Gi, we have to find a small witness of membership of v 
in G2. This can be done in 112 . ^ 



7 Normal form over larger alphabets? 

In Theorem |6.3| we assumed that we are working with an 
alphabet of two letters. Does a similar statement hold for 
alphabets of size 3, 4, . . .? What about alphabets of unfixed 
size? 



For 2 letters, we have generated all multisets generated 
by our grammar from runs D having specific supp(£') us- 
ing two extreme cycles, which led to generating out {D) us- 



A 



mg pairs of extreme cycles in total — Theorem |6.2 
natural conjecture is that a similar normal form exists for 
greater alphabets, except that there would be a polynomial 
(iV^) number of extreme cycles now — this would give us 
a straightforward generalization of Theorem |6.2| and thus 



also of Theorem 6.3 by combining with a generalization 
of Lemma [33] However, this is not true; in fact. Theorem 
6.2 already fails for a three letter alphabet. We present this 
counterexample, because we think it is interesting. 

Theorem 7.1 There exists a context-free grammar G over 
{x, y, z} such that out(G) is not a union of a polynomial 
number of A, B -frames. 

Proof. Consider the following grammar G (in the stan- 
dard commutative grammar notation, with exponential re- 
strictions on the size of productions): 



s - 


^ 


SABCDEz 


A - 




C^D^E^xy 


B - 


■> G2 


D^E^x^y 


C ' 




E^x^y 


D - 


-> E^ 


x^y 


E - 


^ 


x^^y 



The state S generates any number of z's together with 
the same number of ABCDE's. ABCDE generates a 
convex 32-gon on the surface M^^'^'^ (we get 32 corners 
by deciding which transition always to use for each of 
five states A, B, G, D, E; they are points with coordi- 
nates ^ for y e [0..31]). Since we generate z" 
together with {ABCDEy\ out(G) is a cone (i.e., a un- 
bounded pyramid) with 32 edges (each edge is the line 
|j-£j£(£+i) ^ 2;^^ 2}) : z e M} for some y), and hence we 
need more than 16 three-dimensional A, i?-frames to cover 
out(G). This example generalizes to any number of states 
(bigger examples are constructed using the same simple rule 
as the example above) — we need more than 2"^^ A, B- 
frames for a grammar with n states and two transitions 
for each state. Note that the n-state verison of the gram- 
mar above can be written in the limited form (i.e., for each 
derivation s A i, \a\ < 1, \t\ < 2) using 0{n) states. 

Instead of proving that this construction gives a good 
counterexample for each n (i.e., it indeed generates a 2"- 
gon), we present another construction, based on the same 
idea (although we don't get as beatiful grammar as above, 
the proof is simpler). 

For each n G N, we will generate a grammar G„ with n 
states over {x, y}, two transitions for each state, for which 
the set of vertices of the convex hull of out(G) is the set 
of points with coordinates {y{N — y),y) for each odd y e 



7 



[0..N], where N = 2"+^ It is easy to find d; we will 
now show how to construct Gn+i using Gn- We perform 
the following steps. 

• We add x'^ to be always generated right away from 
the start symbol S (i.e., to both transitions from S). 

• Whenever we generate an y using some transition, we 
additionally generate , ignoring (for now) the fact 
that —N is negative. Our vertices are now {f{y),y), 
where f{y) — y{N — y — N) + N"^ for y as before. 
Note that /(y) = {N-y)[N + y). 

• We replace all occurences of y in our grammar with 
a new symbol Y , with two rules: Y v\y~^ ■ Our 
vertices are now still {f{y),y), except that now y is 
now in range [—N..N]. 

• We add y^ to be always generated right away from 
S. Now, our vertices are {g{y),y), for each odd y in 
[0..2iV], and giy) = f{y - N) = {2N - y)y. 

• Thus, we have G„+i, except that our grammar is im- 
proper due to negative transitions. However, it is easy 
to "normalize" our grammar: if it is possible to gen- 
erate, say, from a non-initial state A, add to 
the right side of each transition from A (thus eliminat- 
ing x~°-), and replace each occurence of A on the right 
side of some transition with Ax^°'. Since the gram- 
mar is acyclic, and the initial state S never generates 
a negative number of any terminal, this algorithm wiU 
eventually eliminate all the negative transitions. □ 



Inclusion for fixed alphabets over more 
than 2 letters 



The proof of the generalization Theorem 6.3 to alphabets 
of larger (but still fixed) size is very long and technical. We 
had to omit most proofs for space reasons. 

In this proof, we will require lots of constants; some of 
them are dependant on other. To keep our constants or- 
dered, and make sure that there is no circular reference be- 
tween them, we will name them consistently Ci , C2 , C3 , . . . 
through the whole section; each constant will be defined in 
such a way that it will depend single exponentially on the 
size of the grammar and/or polynomially on the lower num- 
bered constants. By induction, all numbered constants de- 
pend single exponentially on the size of the grammar As 
usual, when we say Ci is polynomial in Cj, we assume that 
the size of alphabet E is fixed. (If |E| is not fixed, then 
Q=0{Cf™), where p is a polynomial.) 



Let Fc\ C [O..C5]|^ (i.e., a set of some linear functions 
over with integer coefficients up to C5) be such that for 
each set of |E| — 1 vertices V C [O..Ci]^, there exists a 
non-zero / e Fc-^ such that fV = 0. This can be done 
with C5 polynomial in Ci . 

Let Lp., — {0, Cs}^ be the set of vertices of the hyper- 
cube of dimension and edge length C3. 

Let TZ{Ci, C3) be the set of functions from Fc^ x Lc^ 
to {-1,0,1}. 

For ar £ TZ, let 



reg(r) 
Reg(r) 

T(r) 



X e 



V/ e Fc.yi e Lc, 

sgn(/w - fl) = Tf^i 

V/ e Fc.yi e Lc, 
sgnifv -fl) e {0,r/,,} 

V/ e Fcyi e Lc, 
sgn{fx) g {0,r/,;} 



The following picture (Figure A) shows what and 
T(r) n Js look like for Ci = 2 and = 3. (If t e T(r), 
then also xt G r(r) for a; > 0; thus, a cross of r(r) and 
gives us information about the whole T(r).) 




By i?®^ we denote {J2beB ^bb 
Let ^ {x gR^ : X >0, \x\ = 



ybe B ate A}. 
!}■ 



Figure A 

The big equilateral triangle is J^. The 19 small white 
circles are points v/\v\ for v e [O..Ci]^. We connect each 
pair of points with a line; these lines correspond to elements 

of Fc,. 

For each r, T{r) n Js is a part of the triangle defined 
by their relationship with each line (above, below, or on 
the line). Thus, each T(r) is either an empty set, or one of 
the points where lines cross (including the 19 circles), or 
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a line segment between two consecutive points where the 
lines cross, or a polygon bounded by lines. 

What does the subdivision of n KJ^ into regions 
(reg and Reg) look like for a large Kl The picture would 
be similar to the picture of T{r) n J^, except that instead of 
each line we would have a bundle of lines corresponding to 
picking different elements of Lc^- In case of |S| = 2 the 
subdivision of into regions is similar to the picture from 
Lemma [374] (the bundles of lines are no longer infinite). 

Lemma 8.1 Let Ci, C3, C4 e N. There exist constants C7 
polynomial in Ci, and C's polynomial in Ci, C3 and C4, 
such that for each r e TZ{Ci, C3), for each v G reg(r), ;/ 
||?;|| > Cs, then v ^ vq + C^t, where t G T(r) n [^..€7]^ 
and Vq € reg(r). 



Note that a I El -dimensional version of Lemma 3.4 fol 



lows easily from Lemma [8T| Lemma [8T| also plays a simi- 
lar role in our proof as Lemma 3.4 did for d — 2. 



Lemma 8.2 Let C'l G N. Then there exists Cg polynomial 
in C7 such that: 

Let P,Q <Z [O.-Cy]^ such that P®"" n Q®' = {0}, and 
^ P,Q. Then there /i a $ G (i.e., a linear function 
over with integer coefficients) such that > 0, $Q < 
0, and |$| < Cg. 

Intuitively, this lemma states that, given two disjoint 
closed convex polygons in some space (in our case, the 
space is Jj], and the polygons are intersections with P®' 
and Q®^), we can separate them strictly with a hyperplane. 
Such separation is a well known property of convex sets; 



Lemma 8.2 gives a polynomial bound on the coefficients of 
such a separating hyperplane. 

Lemma 8.3 Let Ci, C3 G N. Then there exists a constant 
Cii such that: 

Let S = W + y®^, where W C [O..C3f and y C 
[O..Ci]^. Let r G TZ{Ci,C3). Then there exists a matrix 
M G [O..Ci]| such that S n reg(r) = {Wi + MN^) f] 
reg(r), where Wi C [O..Cii]^. 

Theorem 8.4 Let G be a commutative grammar, and let r 
be a region. 

The intersection of out (G) D reg(r) is an intersection 
o/reg(r) and a polynomial union ofGn, Gi-frames, where 
Gil ond Gi are single polynomial in \G\. 

Theorem 8.5 Inclusion is -complete for fixed E. 



Proof of Theorem 8.5 



We apply the methods of Theo- 
rem |6.3| separately for each region from Theorem 8.4 □ 



9 Lower bounds 

For completeness, we provide proofs of lower bounds 
for the complexities of considered problems. These results 
have been previously known (e.g., |Hu84|). 

Theorem 9.1 For a commutative regular grammar G over 
E ( whose size is not fixed), and K G N^, the problem of 
deciding whether K G out(G') is NP-complete. 

Proof. The problem is obviously in NP (the run is 
the witness — the only problem is that it could be larger 
than polynomial by including a large number of transitions 
which produce nothing, but such transitions must form cy- 
cles which can be easily eliminated). We show a reduc- 
tion from the Hamiltonian circuit problem. Let (V, E) be 
a graph. We take Yi = S = V , and for each edge (ui, V2) 
we add a transition vi ^ W2- We pick an initial state so 
and add a final transition so ^ 0. The graph {V, E) has a 
Hamiltonian circuit iff (1, 1, . . .) G out(G). □ 

The same example shows that disjointness is co-NP-hard 
for grammars over alphabets of unfixed size. It is also co- 
NP-complete, since our polynomial algorithm for fixed size 
alphabets can be easily modified to work in co-NP for un- 
fixed size ones. 

Theorem 9.2 For a single letter alphabet E — {a} and 
a commutative grammar G (not necessarily regular), and 
K G N^, the problem of deciding whether K G out(G) is 
NP-complete. 

Proof. The problem is in NP for a similar reason. 
We can reduce the knapsack problem: given a sequence 



of positive integers ki 



, km and K, is there a subset 



L C {1, . . . , m) such that 'YIiI^l ~ Indeed, it is 
easy to produce a grammar of size 0(^,- log(fci)) which 
generates Ka K ^ J^ieL ^'^^ some L. □ 

Theorem 9.3 Let G be a commutative regular grammar 
over E of fixed size. Then the problem of deciding univer- 
sality (out{G) = N^j is coNP-hard even for E — {a}. 

Proof. The problem is in coNP because the witness for 
non-universality is of polynomial length (by the same argu- 
ment as in Theorem [63]). 

We reduce the 3CNF-SAT problem. Let = Ai<j<fe C'^ 
be a 3CNF-formula with n variables xi . . .Xn (which can 
be or 1) and k clauses. Let pi,p2, . ■ . ,Pn be n distinct 
prime numbers. Let i G [l..k]. Suppose that clause Gi is 



of form V 



ke[1..3] ^ak 



Va^. Our grammar will have states 
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Sj, where < j < Mi ~ PaiPa2Pa3', we have cycHc tran- 
sitions Sj A 'S'(j+i)„jo^jv/i' ^] ^ ^ ^^^^ ^ 
fying Vfce[i 3] J mod = Va^- We also have transitions 
So — > 5*0 for each i. 

From simple number theoretic arguments we get that x ^ 
out(G) iff the formula (p is satisfied for Xi — x mod pi. □ 

□ 



Corollary 9.4 Disjointness is coNP- complete for commu- 
tative context-free grammars over E of fixed size, and uni- 
versality, equivalence, and inclusion are coNP -complete for 
commutative regular grammars over T, of fixed size. 



Proof. We get that disjointness and universality for com- 



mutative grammars are coNP-hard from Theorems 9.2 and 
|9.3| respectively. We get the upper bounds by applying the 
same methods as in Theorem |8.5| (or the easier Theorem 
6.3 for alphabets of size 2). In the case of equality and in- 
clusion for regular grammars, we get rid of one level of the 



polynomial hierarchy by using Theorem 5.3 to decide mem- 
bership. □ 



10 Conclusion 

We have shown tight complexity bounds for the prob- 
lems of membership, inclusion (equality), and disjointness 
of Parikh images of regular and context-free languages over 
alphabets of fixed size. 

What about alphabets of unbounded size? Some of the 
problems here remain open; we do not know whether our re- 
sults and methods shed much light on these problems. For 
example, as far as we know, equality of Parikh images of 
both regular and context-free commutative languages (over 
alphabets of unfixed size) is only known to be -hard 
in coNEXPTIME 0Hu85l . Also, for the universality prob- 
lem for commutative context-free grammars over alphabets 
of fixed size, our bounds are not tight: we know that this 
problem is in IIj (as a special case of inclusion), but the 
only lower bound known to us is coNP (from the regular 
version). 

In some places in our paper, it was convenient to use 
grammars which could produce negative quantities of letters 
(Theorem |5.4[ ), or even negative quantities of states (Theo- 
rem |TT]i. It is interesting whether there exists some more 
general theory for such techniques. 

Many thanks to Slawek Lasota for introducing me to 
these problems, and to everyone on our Automata Scientific 
Excursion for the great atmosphere of research. 
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A Equality of context-free grammars for a 
fixed d> 2 — proof details 

Proof of Lemma |8l] Let r eU. 

For all regions reg(r) bounded by some M (which must 
be polynomial), we can take Cs = M, Cr = 0. 

Let Vi be a sequence of elements of reg(r) such that 
Imii^oo \vi\ = oo. Let Wi = Vi/\vi\. Let w be a cluster 
point of Wi. We have w E T{r). Thus, T(r) n Js is non- 
empty. 

Since r(r) is given by linear equations, we get that T(r)n 
Js is a polytope whose vertices (black points where edge 
cross in Figure A) are ti, . . . , t^, where ti e [O.-CgJ^/gi, 
where Qi £ [l.-Cg]. Both Cg and D are bounded polynomi- 
ally. 

Let R* be the set of points v £ Reg(r) which cannot be 
written as vq + t, where t € r(r) and vq is also in Reg(r). 
It can be easily seen that Reg(r) = R* +T{r), and also that 
R* is bounded polynomially by M. 

We will show that our claim is satisfied for Cj = Cg^ 
and Cs = M + Ca^CqD. Let v € reg(r), ||i;|| > Cs- Since 
V e reg(r) C Reg(r), we have v — wqo + J2i ^i^i^ where 
a,; > and vqq e R* . 

Since \ \v\\ > Cs, there must be i such that > C4C6. 
Thus, ai > CiQi. We get our form: v = Vo + C^qiti, where 
Wo is also in reg(r). □ 



Let 1 1 a; 1 1 2 denote the Euclidean 

= ( Je n ®PP) - (Je n ®i 



Proof of Lemma 8.2 

norm of a; e M^. Let X 
since ePP and ePQ are disjoint, ^ X. Let x G X be 
the point of x such that | I2 — min{| |a;||2 : x G X}. 



Now, let Xp,Xq e Je be the points such that Xp- 



Xn 



X. 



Let $ G P^ be such that ^{xp) = 1, ^{xq) = -1, and 
^{z) = for all z £ such that ||z — Xp\\2 — \ \z — Xq\\2- 
This $ satisfies our conditions. We omit the proof that 
this construction indeed works, and that ||$|| is bounded 
polynomially. 

Proof of Lemma 18.31 

Let C4 be the bound on | det M\ for M £ [O..Ci]|. 
Let C7 and Cs be from Lemma [8T| (for our Ci, C3, and 
C4). 

Let Cg be from Lemma [8^ (for our C7). 

Let Cio be such that for each v £ [CCg]^, and each 
P C [O..Ci]^, ifv£ P®', then Xv £ P®[0-Cio] for some 
X £ [O..C4]. 

Let Cii be big enough. 

Let r S Tl{Ci, C3). For r such that reg(r) is bounded 
(by Mq), M — and Cn = Mq. Thus, assume then reg(r) 
is unbounded. 

Let H = {w/\w\ : w £ [O..Ci]^}. yV^ n Js is a 
convex polytope with vertices from H. On the other hand, 
T(r) n Js is a convex polytope bounded by hyperplanes 



going through sets of |E| — 1 vertices from H; moreover, 
it is a minimal such polytope, i.e., it cannot be subdivided 
into two such polytopes of the same dimension by such a 
hyperplane. Thus, either int(T(r)) n Je is disjoint with 
3^P^ n Je (case 1), or r(r) n Js is a subset of 3^P^ n Js 
(case 2). 

In the case (1), there must be a hyperplane separating 
r(r) n Je and yV^ n Je. Let / e Pci be such that 
/(r(r)) > 0, f{t) > for some t £ T(r), and f{y) < 0. 
Since f{t) > 0, we have r{f,l) = 1 for all I £ Lq^- 
Thus, for V £ S n reg(r), we have f{v) > /{wq) for all 
Wo € [O-.Ca]^. On the other hand, for some wq we have 
V = wq + J^Yey ''^yY, thus f{v) < /{wq). A contradic- 
tion. Thus, S n reg(r) = 0. 

In the case (2), there must be a matrix M, whose 
columns are |E| elements of y, such that r(r) C MV^. 

Let V £ S O reg(r). We prove inductively by v. 

If ||w|| < Cii, we are ready. 

Otherwise, using Lemma |8.1| iteratively, we write v as 
Wo+C4(ii + . . .-t-tx), where £ [O..C7]^, and ||uo|| < Cs- 
We have K > (Cn - Cg) /C4C7. 

On the other hand, we can write v as wo+J^Yey ''^yY+ 
Z^YGP"^^^' where wq £ [O-.Ca]^, ny < Cio, my > 
Cio, \P\ < (We get this form just like in the proof of 
Theorem |5.2| ) 

If for some i we have ti £ P®^^ then we are done. 
Indeed, from definition of Cio we have that C4ti — 
J^YeP '^yY, where ay < Cio. On the other hand, C4ti ~ 
J2 M'^Pi, Pi € From the induction hypothesis we can 
present v — CiU in our form F- Thus we can also present v 



Now, what if ^ p®iP? From Lemma let $ be such 
that < 0, $(P) > 0, ||$|| < Cg. We have: 

= $(«o) + C^4^$(iO <C8Cg-C4X 
i 

^(y) = ^{wq) + ^ nyY + ^ myY > 

Yey YeP 

-C3Cgd-\y\CioCiCgd + 

+((Cii-C3 + |:y|CioCi)/Ci) 

This is a contradiction for Cn big enough. 



□ 



Proof of Theorem|831 

Let Ci be the bound o n th e size of a simple cycle, 
i.e., ys C [O..Ci]^ (Lemmali 



6.1 



Let C2 be such that for each run D we have out(P)) = 
out(r'o) + X^ygp'^^-^' where Hi^oll < C2, and P is a 
subset of 3^suppD of size (We get this form and a poly- 
nomial bound for C2 just like in the proof of Theorem 5.2 ) 

Let / be the set of all subsets of S containing at most 
elements. For i £ I, we can create Wi C [O-.Ca]^ so that 
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out(G) = UiG/ Wi + yf'^. The method is similar to the 
one used in the proof of Theorem |6.2| 

Use constants just like in Lemma |831 

Applying Lemma 8.3 to each component of the union, 
we get that for each r, reg(r) n out(G) = reg(r) D 
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